Method and system for assessing vessel obstruction based on machine learning

ABSTRACT

Methods and systems are provided for assessing the presence of functionally significant stenosis in one or more coronary arteries, further known as a severity of vessel obstruction. The methods and systems can implement a prediction phase that comprises segmenting at least a portion of a contrast enhanced volume image data set into data segments corresponding to wall regions of the target organ, and analyzing the data segments to extract features that are indicative of an amount of perfusion experiences by wall regions of the target organ. The methods and systems can obtain a feature-perfusion classification (FPC) model derived from a training set of perfused organs, classify the data segments based on the features extracted and based on the FPC model, and provide, as an output, a prediction indicative of a severity of vessel obstruction based on the classification of the features.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application is a continuation of U.S. patent applicationSer. No. 15/933,854 filed on Mar. 23, 2018, which claims priority fromU.S. Provisional App. No. 62/476,382, filed on Mar. 24, 2017, which areboth herein incorporated by reference in their entireties.

BACKGROUND 1. Field

The present application relates generally to methods and systems toassess a severity of vessel obstruction, and methods and systems to formfeature-perfusion classification (FPC) models that classify trainingfeatures in connection with assessing a severity of vessel obstruction.

2. State of the Art

Coronary artery disease (CAD) is one of the leading causes of deathworldwide. CAD generally refers to conditions that involve narrowed orblocked blood vessels that can lead to reduced or absent blood supply tothe sections distal to the stenosis resulting in reduced oxygen supplyto the myocardium, resulting in, for instance, ischemia and chest pain(angina). A very important aspect in the prevention and treatment of CADis the functional assessment of such narrowed or blocked blood vessels.

Presently, X-ray angiography is the imaging modality used duringtreatment of stenotic (narrowed) coronary arteries by means of aminimally invasive procedure also known as percutaneous coronaryintervention (PCI). During PCI, a (interventional) cardiologist feeds adeflated balloon or other device on a catheter from the inguinal femoralartery or radial artery up through blood vessels until they reach thesite of blockage in the artery. X-ray imaging is used to guide thecatheter threading. PCI usually involves inflating a balloon to open theartery with the aim to restore unimpeded blood flow. Stents or scaffoldsmay be placed at the site of the blockage to hold the artery open.

X-ray angiography is also a standard imaging technique for anatomicalassessment of the coronary arteries and the diagnosis of coronary arterydisease. Although objectivity, reproducibility and accuracy inassessment of lesion severity has improved by means of quantitativecoronary analysis tools (QCA), the physiological significance ofatherosclerotic lesions, which is the most important prognostic factorin patients with coronary artery disease, cannot be appreciated by X-rayangiography.

For intermediate coronary lesions (defined as luminal narrowing of30-70%), for instance, it is not always obvious if the stenosis is arisk for the patient and if it is desired to take action. Overestimationof the severity of the stenosis can cause a treatment which in hindsightwould not have been necessary and therefore exposing the patient torisks that are not necessary. Underestimation of the severity of thestenosis, however, could induce risks because the patient is leftuntreated while the stenosis is in reality severe and actually impedesflow to the myocardium. Especially for these situations it is desired tohave an additional functional assessment to aid in a good decisionmaking.

Fractional Flow Reserve (FFR) has been used increasingly over the last10-15 years as a method to identify and effectively target the coronarylesion most likely to benefit from percutaneous coronary intervention(PCI). FFR is a technique used to measure pressure differences across acoronary artery stenosis to determine the likelihood that the stenosisimpedes oxygen delivery to the heart muscle. The technique involvespercutaneously inserting a pressure-transducing wire inside the coronaryartery and measuring the pressure behind (distal to) and before(proximal to) the lesion. This is best done in a hyperemic state becausein the case of maximum hyperemia, blood flow to the myocardium isproportional to the myocardium perfusion pressure. FFR thereforeprovides a quantitative assessment of the functional severity of thecoronary lesion as described in Pijls et al., “Measurement of FractionalFlow Reserve to Assess the Functional Severity of Coronary ArteryStenoses,” N Engl J Med 1996, 334:1703-1708.

Although the European Society of Cardiology (ESC) and the AmericanCollege of Cardiology/American Heart Association (ACC/AHA) guidelinesrecommend the use of FFR in patients with intermediate coronary stenosis(30-70%), visual assessment, whether or not supported by QCA, of X-raycoronary angiograms alone is still used in over 90% of procedures toselect patients for percutaneous coronary intervention (Kleiman et al,“Bringing it all together: integration of physiology with anatomy duringcardiac catheterization,” J Am Coll Cardiol. 2011; 58:1219-1221).

FFR, however, has some disadvantages. The technique is associated withthe additional cost of a pressure wire which can only be used once.Furthermore, measuring FFR requires invasive catheterization with theassociated cost and procedure time. Also, in order to induce (maximum)hyperemia, additional drug infusion (adenosine or papaverine) isrequired, which is an extra burden for the patient.

Coronary computed tomography (CT) angiography (CCTA) is a non-invasiveimaging modality for the anatomic assessment of coronary arteries butdoes not assess the functional significance of coronary lesions. Due tothe remarkably high negative predictive value of CCTA and itsnon-invasive nature, the main strength of CCTA is its excellent abilityto exclude CAD. Although CCTA can reliably exclude the presence ofsignificant coronary artery disease, many high-grade stenosis seen onCCTA are not flow limiting. This potential for false positive resultshas raised concerns that widespread use of CCTA may lead to clinicallyunnecessary coronary revascularization procedures. This lack ofspecificity of CCTA is one of the main limitations of CCTA indetermining the hemodynamic significance of CAD (Meijboom et al,“Comprehensive assessment of coronary artery stenoses: computedtomography coronary angiography versus conventional coronary angiographyand correlation with fractional ow reserve in patients with stableangina,” Journal of the American College of Cardiology 52 (8) (2008)636-643). As a result, CCTA may lead to unnecessary interventions on thepatient, which may pose added risks to patients and may result inunnecessary health care costs.

Taylor et al “Computational Fluid Dynamics Applied to Cardiac ComputedTomography for Noninvasive Quantification of Fractional Flow Reserve,”Journal of the American College of Cardiology, Vol. 61, No. 22, 2013,and U.S. Pat. No. 8,315,812, describe a noninvasive method forquantifying FFR from CCTA, which we refer to as FFRCT. This technologyuses computational fluid dynamics (CFD) applied to CCTA aftersemi-automated segmentation of the coronary tree including a part of theascending aorta covering the region in which both the left coronaryartery as well as the right coronary artery emanate. Three-dimensional(3D) blood flow and pressure of the coronary arteries are simulated,with blood modeled as an incompressible Newtonian fluid withNavier-Stokes equations and solved subject to appropriate initial andboundary conditions with a finite element method on parallelsupercomputer. The FFRCT is modeled for conditions of adenosine-inducedhyperemia without adenosine infusion. This process is computationallycomplex and time-consuming and may require several hours.

There are several limitations for physiologic assessment of coronarystenosis by FFRCT besides its long computation time as mentioned above.

First, FFRCT is calculated by computational simulation of adenosinemediated hyperemia rather than by actual administration of adenosine.

Second, the value of FFRCT is influenced not only by stenosis severitybut also by the presence of viable or scarred myocardium (De Caterina etal, “Limitations of noninvasive measurement of fractional flow reservefrom coronary computed tomography angiography,” Journal of the AmericanCollege of Cardiology, vol. 59, no. 15, pp. 1408-1410, 2012). The statusof myocardial microvasculature indicates if a certain portion of theheart can be regarded to be healthy. For instance, the presence ofmyocardial ischemia is an indication that a certain portion of the heartis not supplied with enough blood for example due to an (earlier)infarction (FIG. 1). This has an effect on the microvascular resistanceand should be adjusted accordingly in the model calculations.

Third, the calculated FFRCT values may be lower than compared to FFRvalues measured invasively in patients with microvascular disease,because modeling of adenosine-induced hyperemia may overestimate thedegree of vasodilation (Taylor et al, “Computational fluid dynamicsapplied to cardiac computed tomography for noninvasive quantification offractional flow reserve: scientific basis,” Journal of the AmericanCollege of Cardiology, vol. 61, no. 22, pp. 2233-2241, 2013).

Fourth, vascular remodeling and collateral flow are not considered andeven not visible on CCTA, therefore the assumption is made that nocollateral arteries are present which feed the coronary vessel beddistal to the lesion. Collateral flow is an adaptation of the vesselswhere the collateral vessels provide the myocardium with blood bybypassing the stenotic lesion (FIG. 2). The effect of this is that, evenin the case of a very severe stenosis (for instance a total occlusion)the sections distal to the stenosis receive blood flow. Therefore, inpractice the effect of the stenosis is not necessarily severe, and arevascularization is not always needed. When collateral flow is present,this also has an effect on the calculations and should also becompensated. However, due to their small size these collateral vesselsare not commonly visible on CCTA and further steps are needed todetermine the presence of collateral flow.

Fifth, because FFRCT requires accurate anatomic models, numerousartifacts on CCTA may affect FFRCT calculation, such as bloomingartefacts caused by large arterial calcifications and stents. Inaddition, motion, lower SNR, and mis-registration may compromise itsaccuracy. Therefore, CCTA data with good image quality is essential forthe accuracy of FFRCT interpretation.

In order to keep the computational demands on a feasible level a reducedmodel can be used in the calculation. Specifically, sections of thecoronary tree can be represented by a one-dimensional network orzero-dimensional (lumped) model. This multi-scale approach was adoptedby Kim et al, “Patient-specific modeling of blood flow and pressure inhuman coronary arteries,” Annals of Biomedical Engineering 38,3195-3209, 2010 to compute physiologically realistic pressure and flowwaveforms in coronary vessels at baseline conditions.

Nickisch et al. “Learning patient-specific lumped models for interactivecoronary blood flow simulations”, International Conference on MedicalImage Computing and Computer-Assisted Intervention, Springer, 2015, pp.433-441, presents a technique to estimate FFR in the coronary arterytree from a CCTA scan, based on blood flow simulations using aparametric patient specific lumped model. This technique is designed tofurther reduce computational demands. In the aforementioned publicationthe authors use a hydraulic system analogy to model the coronary treewith an electrical circuit interpretation where volumetric flow rate wasmodeled as an electrical current and pressure in the coronary artery asa voltage. This technique achieved high accuracy and real-time feedback,but it strongly depends on the segmentation of the coronary artery treeand determination of its centerline. Moreover, the method requiresfurther clinical validation as it was only validated on a small set ofCCTA scans.

A different approach to reduce the computation time required by CFD, isintroduced in WO2015/058044. In this work, a method is disclosed toassess the FFR by means of a machine learning system which is based onfeatures extracted from the anatomical three-dimensional coronarygeometry. The machine-learning is trained by using geometric extractedfeatures from synthetically generated 3D stenosis geometries and FFRvalues corresponding to the synthetically generated 3D stenosis computedby use of CFD. After the learning phase, the system predicts the FFRbased on extraction of the same features of an unseen anatomicalthree-dimensional coronary geometry which is for instance extracted fromCCTA by means of image segmentation methods.

A similar approach is disclosed in US2014/0073977 for assessment of FFRby means of machine-learning algorithm on geometrical features extractedfrom three-dimensional vessel geometry. In this method the machinelearning was performed by extracted 3D coronary tree geometries frompatient image data and FFR values corresponding to the patientgeometries were computed by CFD.

All of the above described methods heavily rely on the anatomical vesselgeometry extracted from the patient's image data. This involves, forassessment of FFR in coronaries, the segmentation of coronary tree. Thedemands on the segmentation accuracy are high, especially for stenoticsegments. Taking into account that a mild coronary obstruction has anaverage diameter between 1.5-2.5 mm and that spatial resolution of CCTAis in the range of 0.25 mm isotropic, obtaining accurate 3D morphologyby means of segmentation is a very challenging task. This in addition tothe imaging artifacts induced by calcified coronary atheroscleroticlesions, or other imaging artifacts as discussed before.

Methods to assess the functional significance of a coronary lesion thatdo not rely on the anatomical coronary vessel geometry combined withblood flow modeling have been developed. For example, George et al. in“Adenosine stress 64-and 256-row detector computed tomographyangiography and perfusion imaging a pilot study evaluating thetransmural extent of perfusion abnormalities to predict atherosclerosiscausing myocardial ischemia,” Circulation: Cardiovascular Imaging 2 (3)(2009) 174-182, demonstrated that comparison of myocardial regionsimaged at rest and pharmacologically-induced stress by administration ofadenosine, reveals areas with perfusion defects, which are directlycaused by hemodynamically significant stenosis. Although this approachis promising as it merges the anatomic information of CCTA withfunctional analysis, it requires an additional CT scan which inevitablyleads to higher radiation dose and longer examination time and the needfor injection of pharmacological stress agents.

There is thus the need for a patient specific method to identifypatients with functional significant stenosis in one or more coronaryarteries based on the information extracted from a single CCTA datasetonly, which has low computational complexity demands and which takesinto account the status of the myocardial microvasculature andcollateral flow, without relying on the detailed morphology of thecoronary arterial system.

SUMMARY OF THE INVENTION

At least one aim of CCTA is to identify cardiac and coronary arteryanatomy by means of injecting an exogenous contrast agent, usually viaintravenous injection in an antecubital vein to enhance the cardiacand/or coronary anatomy during imaging. According to the Society ofCardiovascular Computed Tomography Guidelines for the performance andacquisition of CCTA as described by Abbara et al. in “SCCT guidelinesfor the performance and acquisition of coronary computed tomographicangiography: A report of the Society of Cardiovascular ComputedTomography Guidelines Committee Endorsed by the North American Societyfor Cardiovascular Imaging (NASCI),” J Cardiovasc Comput Tomogr. 2016November-December; 10(6):435-449, the contrast medium injection is timedin such a way that the coronary arterial system contains sufficientcontrast medium to clearly distinguish the coronary artery lumen fromsurrounding soft tissues. This enables the physician assessment ofluminal narrowing as well as coronary artery stenosis with an optimalimage quality and thus accuracy. To ensure adequate coronary arteryopacification, the aforementioned guidelines describes that CCTA imageacquisition is typically started once a pre-defined thresholdattenuation value has been reached in a pre-defined anatomical structure(most often this concerns the descending aorta), or by waiting a certaindelay time after enhancement is first visible in the ascending aorta.The inventors have furthermore recognized that due to above describedmethod the acquisition of CCTA provides contrast enhancement in theventricular myocardium, since the injected contrast medium, once it ispresent in the coronary arteries, will also be delivered to successivelysmaller generations of coronary arterioles from where it traverses intothe coronary microvasculature, which will lead to (subtle) enhancementof the myocardium.

Functionally significant coronary artery stenosis causes ischemia in theventricular myocardium. Due to the above described acquisitionproperties of CCTA, there is a difference in myocardial texturecharacteristics between normal and ischemic parts of the myocardium atthe time of CCTA image acquisition.

It is therefore an objective of the present application to perform apatient classification based on machine learning using features of themyocardium.

In embodiments herein, methods and systems are described that present anovel manner to automatically identify patients with functionallysignificant stenosis in at least one coronary artery, based on theinformation extracted from a single CCTA dataset. While more than oneCCTA dataset may be utilized, only a single dataset is needed. Themethod first segments the myocardium using for instance a multi-scaleCNN trained on manually annotated data. Thereafter, to characterize theleft ventricle myocardium, the myocardium characteristics can bederived. This can be done by feature-engineering or by e.g.convolutional autoencoder, or a combination thereof.

Once the characteristics of the myocardium have been extracted, thepatients can be classified into those with or without functionallysignificant stenosis based on these characteristics. This can be donewith any classifier (supervised or unsupervised). In a preferredembodiment, patients are classified with a support vector machine (SVM)classifier into those having functionally significant stenosis in one ormore of the coronary arteries and those without it, according toinvasively determined FFR measurements that are the current referencestandard.

In accordance with aspects herein, a method is provided for assessingthe presence of functionally significant stenosis in one or morecoronary arteries, further known as a severity of vessel obstruction.The method assesses if patients suffers from significant coronaryobstructions, without differentiating which particular obstruction isresponsible for a particular functional significant coronaryobstruction. The method can implement a prediction phase that includes:obtaining a contrast enhanced volume image dataset for a target organ;segmenting at least a portion of the volume image data set into datasegments corresponding to wall regions of the target organ; analysingthe data segments to extract features that are indicative of an amountof perfusion experiences by wall regions of the target organ; obtaininga feature-perfusion classification (FPC) model derived from a trainingset of perfused organs; classifying of the data segments based on thefeatures extracted and based on the FPC model; and providing, as anoutput, a prediction indicative of a severity of vessel obstructionbased on the classification of the features.

In accordance with aspects herein, the FPC model can represent therelationship between training features and reference fluidodynamicparameters indicative of baseline amounts of vessel perfusion forcorresponding wall regions of the training set of perfused organs.

In accordance with aspects herein, the reference fluidodynamic parametercan represent an invasive fractional flow reserve measurement.

In accordance with aspects herein, the features can be texture and/ormorphologic features.

In accordance with aspects herein, the features can be determined usinga convolutional auto-encoder, Gaussian filters, transmural perfusionratio, Haralick features, myocardium thickness or shape of the targetorgan.

In accordance with aspects herein, the organ can be the myocardium andthe vessels the coronary arteries.

In accordance with aspects herein, the classifying operation can utilizesecondary information to perform the classification. The secondaryinformation can include one or more of the following parameters:coronary tree anatomy, demographic information of the patient, coronaryartery calcification, coronary plaque, spectral multi-energy or photoncounting, ECG parameters, cardiac biomarkers, adipose tissue surroundingor within the heart, shape of myocardium, or the like.

In accordance with aspects herein, the analyzing operation can includeextracting, for each of the data segments, a feature vector thatcomprises multiple factors that are measured or extracted from thecorresponding data segment, wherein the multiple factors describe orcharacterize a nature of the corresponding wall region.

In accordance with aspects herein, the FPC model can be obtained from adatabase of contrast enhanced volume image data sets and associatedtraining feature vectors extracted from the contrast enhanced volumeimage data sets, where the training feature vectors include knownlabels. The classifying operation can utilize a machine-learningalgorithm that is trained based on the known labels, where themachine-learning algorithm classifies the data segments based on thefeatures.

In accordance with aspects herein, the indication that is output canindicate severity of the vessel obstruction relative to a trainingvessel obstruction.

In accordance with aspects herein, the method further comprisesimplementing a training phase to form the FPC model that classifiestraining features for the training set of perfused organs from contrastenhanced volume image datasets of the organ of the training set and areference fluidodynamic parameter related to a vessel or vesselsperfusing the organs. The training phase can include: providing contrastenhanced volume image datasets of each of the organs in the trainingset; segmenting the organs of the training set; analysing the datasegments to extract training features that are indicative of an amountof perfusion experiences by wall regions of the organs of the trainingset; and classifying the training features of the organs of the trainingset relative to reference fluidodynamic parameters indicative ofbaseline amounts of vessel perfusion for corresponding regions of thetraining set of perfused organs to form the FPC model.

In accordance with aspects herein, the method can further includeclustering the features or training features extracted before performingthe classifying operations in the training phase and/or in theprediction phase.

In accordance with aspects herein, the analyzing can extract a featurevector comprising a series of factors, where each of the factors has avalue representing an amount of variation in a characteristic ofinterest over multiple clusters.

In accordance with aspects herein, a method is provided to form afeature-perfusion classification (FPC) model that classifies trainingfeatures in connection with assessing a severity of vessel obstruction.The method can include: a) obtaining a contrast enhanced volume imagedataset for a training perfused organ; b) segmenting at least a portionof the volume image data set into data segments corresponding to wallregions of the perfused target organ; c) analysing the data segments toextract training features that are indicative of an amount of perfusionexperiences by wall regions of the training perfused organ; d)classifying the training features of the training perfused organrelative to reference fluidodynamic parameters indicative of baselineamounts of vessel perfusion for corresponding regions of the trainingperfused organ to form the FPC model.

In accordance with aspects herein, a system is provided for assessing aseverity of vessel obstruction, which includes: memory configured tostore a contrast enhanced volume image dataset for a target organ; oneor more processors that, when executing program instructions stored inthe memory, are configured to: a) segment at least a portion of thevolume image data set into data segments corresponding to wall regionsof the target organ; b) analyse the data segments to extract featuresthat are indicative of an amount of perfusion experiences by wallregions of the target organ; c) obtain a feature-perfusionclassification (FPC) model derived from a training set of perfusedorgans; d) classify of the data segments based on the features extractedand based on the FPC model; and e) provide, as an output, a predictionindicative of a severity of vessel obstruction based on theclassification of the features.

In accordance with aspects herein, the FPC model can represent arelationship between training features and reference fluidodynamicparameters indicative of baseline amounts of vessel perfusion forcorresponding regions of the training set of perfused organs

In accordance with aspects herein, the reference fluidodynamic parametercan represent an invasive fractional flow reserve measurement.

In accordance with aspects herein, the one or more processors can beconfigured to perform classifying operation by utilizing secondaryinformation to perform the classification. The secondary information caninclude one or more of the following parameters: coronary tree anatomy,demographic information of the patient, coronary artery calcification,coronary plaque, spectral multi-energy or photon counting, ECGparameters, cardiac biomarkers, adipose tissue surrounding or within theheart, shape of myocardium, or the like.

In accordance with aspects herein, the one or more processors can beconfigured to perform the analyzing operation by extracting, for each ofthe data segments, a feature vector that comprises multiple factors thatare measured or extracted from the corresponding data segment, whereinthe multiple factors describe or characterize a nature of thecorresponding region.

In accordance with aspects herein, the one or more processors can beconfigured to extract a feature vector comprising a series of factors,where each of the factors has a value representing an amount ofvariation in a characteristic of interest over multiple clusters.

In accordance with aspects herein, the one or more processors can beconfigured to extract a feature vector comprising a series of factors,where each of the factors represents an intensity of a characteristic ofinterest over multiple segments of the myocardium.

In accordance with aspects herein, the one or more processors can beconfigured to extract a feature vector comprising a series of factors,where a subset of the factors in the series represent intensity withinsegments, and where another subset of the factors in the seriesrepresent values indicative of myocardium volume, minimum myocardiumthickness and/or maximum myocardium thickness.

BRIEF DESCRIPTION OF THE DRAWINGS

The characteristics of the present application and the advantagesderived therefrom will be more apparent from the following descriptionof non-limiting embodiments, illustrated in the annexed drawings.

FIG. 1 shows an example of ischemia due to a stenosis of an artery.

FIG. 2 shows an example of collateral flow across a lesion.

FIG. 3 shows illustrates a flowchart of a machine learning based methodfor determining the presents of functional significant stenosis in oneor more coronary arteries to an embodiment of the present invention.

FIG. 4 shows a functional block diagram of an exemplary CT system.

FIG. 5 shows an example of an end result of a Left Ventricle myocardiumsegmentation.

FIG. 6 shows a flowchart of the generation of the feature-perfusionclassification model as performed by the training phase.

FIG. 7 shows an example of the architecture of a convolution autoencoder (CAE) as used during learning the CAE.

FIG. 8 shows six examples of reconstructed patches. Each row contains 2pairs of patches, each pair contains the original input patch (right)and the reconstructed output patch (left).

FIG. 9 shows an example of LV myocardium clustering. Different greyvalues represent different clusters.

FIG. 10 shows an example of coronary anatomy.

FIG. 11a shows an example of coronary calcium scoring. The white arrowsidentify coronary calcium within a CTTA dataset.

FIG. 11b shows an example of detection of coronary calcium scoringwithin a CCTA dataset. The white arrows point to the regions which areidentified as coronary calcium.

FIG. 12 shows an example of a normal ECG and an ECG with an elevated STsegment.

FIG. 13 shows an example of dilated cardiomyopathy.

FIG. 14 shows an example of a feature space wherein the training data isrepresented as points, mapped so that the data of the separatecategories are divided by a clear gap that is as wide as possible.

FIG. 15 shows a flowchart of an embodiment of the present applicationfor the prediction phase.

FIG. 16 shows an example of the architecture of a convolution autoencoder (CAE), as used during the prediction phase.

FIG. 17 shows an illustration of the classifier to classify unseen data.Within this visual representation, the input is the feature vectorcomputed from the unseen image and the output two classes.

FIG. 18 shows a high-level block diagram of an example of a CT system.

DETAILED DESCRIPTION OF EMBODIMENTS

The term “unseen”, as used throughout, refers to “non-training” items.For example, an unseen image is not a training image, an unseen featureis not a training feature. Instead, the unseen features, images,geometries and other unseen items refer to aspects of a patient orobject of interest that is being analysed during the prediction phase ofoperation.

The present application relates to a method and system for machinelearning to assess the hemodynamic functional severity of one or morevessel obstructions of a target organ based on contrast enhancedvolumetric image dataset. In a preferred embodiment, the target organrepresents the myocardium and the vessels the coronary arteries. Afunctionally significant stenosis is a hemodynamically significantobstruction of a vessel, and with respect to coronary arteries itdefines the likelihood that coronary artery obstruction(s) impedesoxygen delivery to the heart muscle and causes anginal symptoms.Fractional flow reserve is a hemodynamic index for assessment offunctionally significant coronary artery obstruction(s). In addition tofractional flow reserve, other hemodynamic indices can be used to assessfunctionally significant coronary artery obstruction(s), such ascoronary flow reserve, instantaneous wave-free ratio, hyperemicmyocardium perfusion, index of microcirculatory resistance and pressuredrop along a coronary artery.

Embodiments of the present application utilize machine learning todetermine the presents of functional significant stenosis in one or morecoronary arteries from a CCTA dataset. Machine learning is a subfield ofcomputer science that “gives computers the ability to learn withoutbeing explicitly programmed”. Evolved from the study of patternrecognition and computational learning theory in artificialintelligence, machine-learning explores the study and construction ofalgorithms that can learn from and make predictions on data—suchalgorithms overcome following strictly static program instructions bymaking data driven predictions or decisions, through building a modelfrom sample inputs. Machine-learning is employed in a range of computingtasks where designing and programming explicit algorithms is infeasible.

Given a dataset of images with known class labels, machine-learningsystem can predict the class labels of new images. There are at leasttwo parts to any such system. The first part of the machine-learning isa feature extraction (extractor), being an algorithm for creating afeature vector given an image. A feature vector comprises a series offactors (e.g. multiple numbers) that are measured or extracted from theimage dataset(s), which describe or characterize the nature of thecorresponding wall region of the image. These features are then used bythe second part of the system, a classifier, to classify unseen featurevectors extracted from the unseen image. Given a (large) database ofimages and extracted feature vectors whose labels are known and wereused beforehand to train the machine-learning algorithm, classifyingunseen images based on the features extracted the same way as in imageswith (known) labels (training images) is possible.

FIG. 3 shows a flow chart illustrating the operations according to anembodiment of the present application. The operations employ an imagingsystem capable of acquiring and processing CCTA dataset of an organ (orportion thereof) or other object of interest.

FIG. 4 is a functional block diagram of an exemplary CT system, whichincludes a CT imaging apparatus 112 that operates under commands fromuser interface module 116 and will provide data to data processingmodule 114.

The CT imaging apparatus 112 captures a CT scan of the organ ofinterest. The CT imaging apparatus 112 typically includes an X-raysource and multi detector mounted in a rotatable gantry. The gantryprovides for rotating the X-ray source and detector at a continuousspeed during the scan around the patient who is supported on a tablebetween the X-ray source and detector.

The data processing module 114 may be realized by a personal computer,workstation or other computer processing system. The data processingmodule 114 processes the CT scan captured by the CT imaging apparatus112 to generate data as described herein. The user interface module 116interacts with the user and communicates with the data processing module114. The user interface module 116 can include different kinds of inputand output devices, such as a display screen for visual output, a touchscreen for touch input, a mouse pointer or other pointing device forinput, a microphone for speech input, a speaker for audio output, akeyboard and/or keypad for input, etc. The data processing module 114and the user interface module 116 cooperate to carry out the operationsof the processes described herein.

The data processing module 114 includes one or more memory 118 and oneor more processors 120. The memory 118 stores, among other things, thecontrast enhanced volume dataset for the target organ, data segments,features extracted from analysis of the data segments, one or more FPCmodels, classifications for the data segments and predictions indicativeof a severity of vessel obstruction. The memory 118 may also store oneor more contrast enhanced volume datasets for the training perfusedorgans, data segments corresponding to the wall regions of the perfusedtarget organs, training features extracted from analysis of the trainingdata segments, classifications for the training features, one or moreFPC models, and known labels. The memory 118 also stores software codethat directs the one or more processors 120 to carry out the operationsof the processes described herein. For example, the memory 118 mayinclude an optical disc or other form of persistent memory such as a USBdrive or a network server. The software code can be directly loadableinto the memory of a data processing module 114 for carrying out theoperations described herein.

In accordance with aspects herein, the imaging system has previouslyacquired and stored at least one CCTA dataset of an object of interest.Any imaging device capable of providing a CT scan can be used for thispurpose. In accordance with aspects herein, the one or more processors120 of the data processing module 114 implement a method for assessing aseverity of vessel obstruction, the method implementing a predictionphase that comprises: obtaining a contrast enhanced volume image datasetfor a target organ; segmenting at least a portion of the volume imagedata set into data segments corresponding to wall regions of the targetorgan; analysing the data segments to extract features that areindicative of an amount of perfusion experiences by wall regions of thetarget organ; obtaining a feature-perfusion classification (FPC) modelderived from a training set of perfused organs; classifying of the datasegments based on the features extracted and based on the FPC model; andproviding, as an output, a prediction indicative of a severity of vesselobstruction based on the classification of the features.

The memory 118 may store one or more FPC models 122 and referencefluidodynamic parameters 124. The FPC models 122 include a relationshipbetween training features and the reference fluidodynamic parameters 124indicative of baseline amounts of vessel perfusion for correspondingwall regions of the training set of perfused organs. For example, therelationship may be “functionally significant stenosis present” or “nosignificant stenosis present”. In case of two classes. Optionally,non-limiting examples of other relationships include “no functionallysignificant stenosis present”, “mild functionally significant stenosispresent” or “severe functionally significant stenosis present.” Thereference fluidodynamic parameters 124 may comprise an invasivefractional flow reserve measurement. The memory 118 stores the features126 that are extracted from the data segments. The features 126 textureand/or morphologic features that are descriptive of a texture ormorphology of the corresponding wall region.

For example, the processors 120 may determine the features using aconvolutional auto-encoder, Gaussian filters, transmural perfusionratio, Haralick features, myocardium thickness or shape of the targetorgan. The processors 120 may perform the classifying operationutilizing secondary information. For example, the secondary informationmay comprise one or more of the following parameters: coronary treeanatomy, demographic information of the patient, coronary arterycalcification, coronary plaque, spectral multi-energy or photoncounting, ECG parameters, cardiac biomarkers, adipose tissue surroundingor within the heart, shape of myocardium, or the like.

The processors 120 perform the analyzing operation by extracting, foreach of the data segments, a feature vector that comprises multiplefactors that are measured or extracted from the corresponding datasegment, wherein the multiple factors describe or characterize a natureof the corresponding wall region. The processors 120 obtain the FPCmodel from a data base of contrast enhanced volume image data sets andassociated training feature vectors extracted from the contrast enhancedvolume image data sets, the training feature vectors including knownlabels, wherein the classifying operation utilizes a machine-learningalgorithm that is trained based on the known labels, themachine-learning algorithm classifying the data segments based on thefeatures.

The processor 120 implement a training phase to form the FPC model thatclassifies training features for the training set of perfused organsfrom contrast enhanced volume image datasets of the organ of thetraining set and a reference fluidodynamic parameter related to a vesselor vessels perfusing the organs, the training phase comprising:providing contrast enhanced volume image datasets of each of the organsin the training set; segmenting the organs of the training set;analysing the data segments to extract training features that areindicative of an amount of perfusion experiences by wall regions of theorgans of the training set; and classifying the training features of theorgans of the training set relative to reference fluidodynamicparameters indicative of baseline amounts of vessel perfusion forcorresponding regions of the training set of perfused organs to form theFPC model.

The processor 120 cluster the features or training features extractedbefore performing the classifying operations in the training phaseand/or in the prediction phase. In accordance with aspects herein, theprocessor 120 extract, as the features, a feature vector comprising aseries of factors, where each of the factors has a value representing anamount of variation in a characteristic of interest over multipleclusters.

In accordance with aspects herein, the processors 120 form afeature-perfusion classification (FPC) model that classifies trainingfeatures in connection with assessing a severity of vessel obstruction,the method comprises: a) obtaining a contrast enhanced volume imagedataset for a training perfused organ; b) segmenting at least a portionof the volume image data set into data segments corresponding to wallregions of the perfused target organ; c) analysing the data segments toextract training features that are indicative of an amount of perfusionexperiences by wall regions of the training perfused organ; d)classifying the training features of the training perfused organrelative to reference fluidodynamic parameters indicative of baselineamounts of vessel perfusion for corresponding regions of the trainingperfused organ to form the FPC model.

In accordance with aspects herein, the processors 120 assess a severityof vessel obstruction, by executing program instructions stored in thememory, to: a) segment at least a portion of the volume image data setinto data segments corresponding to wall regions of the target organ; b)analyse the data segments to extract features that are indicative of anamount of perfusion experiences by wall regions of the target organ; c)obtain a feature-perfusion classification (FPC) model derived from atraining set of perfused organs; d) classify of the data segments basedon the features extracted and based on the FPC model; and e) provide, asan output, a prediction indicative of a severity of vessel obstructionbased on the classification of the features.

In accordance with aspects herein, the processors 120 are configured toperform the analyzing operation by extracting, for each of the datasegments, a feature vector that comprises multiple factors that aremeasured or extracted from the corresponding data segment, wherein themultiple factors describe or characterize a nature of the correspondingregion. In accordance with aspects herein, the processors 120 areconfigured to extract, as the features, a feature vector comprising aseries of factors, where each of the factors has a value representing anamount of variation in a characteristic of interest over multipleclusters. In accordance with aspects herein, the processors 120 areconfigured to extract, as the features, a feature vector comprising aseries of factors, where each of the factors represents an intensity ofa characteristic of interest over multiple segments of the myocardium.In accordance with aspects herein, the processors 120 are configured toextract, as the features, a feature vector comprising a series offactors, where a subset of the factors in the series represent intensitywithin segments, and where another subset of the factors in the seriesrepresent values indicative of myocardium volume, minimum myocardiumthickness and/or maximum myocardium thickness.

The operations of FIG. 3 can also be carried out by software code thatis embodied in a computer product (for example, an optical disc or otherform of persistent memory such as a USB drive or a network server). Thesoftware code can be directly loadable into the memory of a dataprocessing system for carrying out the operations of FIG. 3.

In this example it is assumed that the imaging system has acquired andstored at least one CCTA dataset of an object of interest. Any imagingdevice capable of providing a CT scan can be used for this purpose.

The present application is particularly advantageous in myocardiumanalysis based on CCTA dataset and it will mainly be disclosed withreference to this field, particularly for patient classification.

An embodiment of the present application is now disclosed with referenceto FIG. 3. The therein-depicted steps can, obviously, be performed inany logical sequence and can be omitted in parts.

As described in step 301 of FIG. 3, a contrast enhanced CCTA dataset isobtained. This CCTA dataset can be obtained from a digital storagedatabase, such as a picture archiving and communication system (PACS) ora VNA (vendor neutral archive), a local digital storage database, acloud database, or acquired directly from a CT imaging modality. Duringthe CCTA imaging, a contrast agent was induced in the patient.Furthermore, the CCTA imaging can be ECG triggered.

To identify patients with functionally significant stenosis, the leftventricle (LV) and/or right ventricle (RV) myocardium wall needs to besegmented in the CCTA dataset as depicted in step 302 of FIG. 3. Thiscan be done manually by the user or by (semi)automatic segmentation. Oneexample of an automatic segmentation of the LV myocardium is given byZreik et. al., “Automatic segmentation of the left ventricle in cardiacct angiography using convolution neural networks,” 2016 IEEE 13^(th)International Symposium on Biomedical Imaging (ISBI), 2016, pp 40-43.Zreik et. al. discloses a method in which the myocardium isautomatically segmented using a convolutional neural network (CNN)trained on manually annotated data. At the end of this step asegmentation of the myocardium is present. An example of such amyocardium segmentation is shown in FIG. 5, which shows the segmented LVmyocardium in an axial, sagittal and coronal CCTA image slice.

Functionally significant coronary artery stenosis causes ischemia in themyocardium which impacts the texture characteristics of the myocardiumwall in a CCTA dataset. Hence, by describing the myocardium, ischemicchanged could be captured. Step 303 of FIG. 3, the featurescharacterizing the myocardium are computed from the CCTA dataset. Tolearn the features characterizing the myocardium, machine-learning isutilized. Step 304 of FIG. 3 a machine learned feature-perfusionclassification model is obtained. This model is generated during atraining phase. In the training phase the relationship between featuresand a reference standard is learned. The reference value can represent afluidodynamic parameter, such as FFR. The features extracted in step 303of FIG. 3 are designed to recognize (hidden) patterns, such as texturepatterns, within the myocardium that correlates to functionallysignificant coronary obstruction(s). This design of extraction featuresdirectly from the myocardium covers two important elements for theassessment of hemodynamic significant lesion(s) that are not consideredin prior art. First, collateral arteries, which provide blood flow tothe myocardium by bypassing the obstruction (FIG. 2) is automaticallytaken into consideration. Second, myocardium microvasculature diseasesuch as presence of ischemia (FIG. 1) is automatically taken intoconsideration.

Step 305 of FIG. 3 assessed the severity of coronary vesselobstruction(s) of an unseen CCTA dataset by classifying the unseen CCTAdataset (prediction phase) based on the learned feature-perfusion modelwithin step 304 of FIG. 3. The output (step 306 of FIG. 3) is aprediction indicative of the likelihood of a functional significantcoronary obstruction is present.

FIG. 6 illustrates a framework for implementing the feature-perfusionclassification model as described by step 304 of FIG. 3. FIG. 6illustrates the training phase of the system. In step 601 of FIG. 6 thereference standard is obtained as used to train the feature-perfusionclassification model. The reference standard is a database whichcontains data of multiple patients. Each set within this databasecontains for each patent a) contrast enhanced CT datasets (step 602)with belonging b) invasively measured fractional flow reserve referencevalues (step 603). Features extracted during the training phase are alsoknown as training features.

Step 604 of FIG. 6 segments the myocardium wall and is identical to step302 of FIG. 3. Block 303 within FIG. 6 represents step 303, thedefinition of the features, of FIG. 3 and will be explained in moredetail with reference to FIG. 6.

Within step 605 of FIG. 6, the features characterizing the myocardiumare extracted from the CCTA dataset. Even though the myocardium iscontrast enhanced, these changes are mostly subtle and it would beextremely challenging—if not impossible in the case of smaller perfusiondefects—to manually label myocardial voxels affected by ischemia. In apreferred embodiment, the myocardium is characterized by the features inan unsupervised manner, extracted via encodings, as determined by aconvolutional autoencoder (CAE). Alternative, any other engineeredcharacteristic that describes myocardium texture (e.g. Gaussian filters,Haralick texture features) and/or morphology (e.g. myocardium thickness,myocardium volume, ventricular volume but also shape of the heartitself) can be used as features. An example of such alternativeengineered feature method designed to quantify the perceived texture ofthe myocardium is by computing Haralick texture features, which capturesnumerical features of a texture using spatial relations of similar graytones (Robert M. Haralick et al., “Textural Features for ImageClassification,” IEEE Transactions on Systems, Man, and Cybernetics,1973, SMC-3 (6): 610-621). Another example of such an alternativeengineered feature method is the transmural perfusion ratio whichdefines the ratio of perfusion (or Hounsfield values) between theendocardial and epicardial layers of the myocardium (Arbab-Zadeh et al.,“Adenosine stress 64- and 256-row detector computed tomographyangiography and perfusion imaging: a pilot study evaluating thetransmural extent of perfusion abnormalities to predict atherosclerosiscausing myocardial ischemia,” Circ Cardiovasc Imaging. 2009 May;2(3):174-82). Any combination of these features can be selected. If thefeatures are local (e.g. per voxel, supervoxel or in same way definedcluster of voxel), dimensionality reduction is needed to represent thepatient myocardium instead of its voxels, for instance by clustering andusing higher order statistics on the cluster as features.

In a preferred embodiment, the myocardium is characterized by thefeatures in an unsupervised manner, extracted via encodings, asdetermined by a CAE (Goodfellow, et. al., “Deep Learning (AdaptiveComputation and Machine Learning series),” Nov. 18, 2016, ISBN 10:0262035618). A CAE compress all the data from an input image to a smallvector from which it must contain enough information to reconstruct theinput image by the decoder. By this the autoencoder is forced to learnfeatures about the image being compressed. A typical CAE contains of twomajor parts, an encoder and a decoder. The CAE compresses (encodes) thedata to lower dimensional representations by convolutional operationsand max-pooling, and subsequently expands (decodes) the compressed formto reconstruct the input data by deconvolutional operations andunpooling.

The CAE architecture used in this embodiment is shown in FIG. 7. The CAEarchitecture presented in FIG. 7 should be considered as an example, andother CAE architectures can be deployed. The encoder (711) e.g.comprises of one convolutional and one max-pooling layer, compressingthe input and one fully connected layer providing the encodings. Thedecoder (709) comprises of one fully connected layer, as well as oneunpooling and one deconvolution layers providing the reconstructedinput. Nonlinearity is applied to the output of each layer to enable theCAE to handle more complex data. For example, Exponential Linear Unit(ELU) (Clevert et al., “Fast and accurate deep network learning byexponential linear units (ELUs),” in: International Conference onLearning Representations, 2016”) can be used as nonlinear activationfunction in all layers except the output layer, where nonlinearity isnot applied.

The detailed CAE architecture, as presented by FIG. 7, has the followingdesign: The size of the input of the CAE was set to 48×48 voxelspatches. The CAE comprised of one convolutional layer with 16 kernels of5×5 (702) and one 2×2 max-pooling layer (703), followed by the encodinglayer of one fully connected layer with 512 units (704 FIG. 7,representing the number of encodings, N). The generated encodings servedas the input of the decoder part, which comprised of one fully connectedlayer with 10816 units (705), one 2×2 unpooling layer (706) followed byone deconvolution layer with a single 5×5 kernel (707). Other parameterscan be used, for instance different kernel sizes, number of encodinglayers as well as other CAE architectures and activation functions.Furthermore, the input size of the CAE can be different as well of theuse of a 3D input patch (volume), or even 4D input patch for instance tosupport multiphase CCTA datasets. The input patch can even be a higherdimensional input patch, for instance n-dimensional to supportmulti-energy CCTA dataset. Learning of the CAE is performed in anunsupervised manner such that extracted features (encodings in case ofCAE) describe and characterize the myocardium within the CCTA dataset.This unsupervised learning of the CAE is performed by using the CCTAdatasets (602) within the reference standard (601). From each CCTAdataset patches are extracted around randomly selected myocardium voxelswithin the segmented myocardium as a result of step 604.

During training, the CAE compress (encodes) input image (701) to a smallvector of numbers (encodings, 710) and subsequently expands (decodes)the compressed form (output, 708) to reconstruct the input image. TheCAE is trained by comparing the reconstructed output image (708) and theinput image (701) in an iterative process to minimize the differencebetween them. The difference between the input image (input of theencoder, 701 of FIG. 7) and the reconstructed output image (output ofthe decoder, 706 of FIG. 7) may be determined by the mean squared errorand is iteratively minimized by for instance the Stochastic GradientDescent (SGD) with Nesterov momentum as described by Y. Nesterov, et al.“Gradient methods for minimizing composite objective function,” Tech.rep., UCL (2007). In each iteration, convolutional filters (702 and 703)are adjusted and updated. This iterative process stops when the meansquared error is within a predefined value. The result of this processensures that abstract features (encodings) are produced from the inputimage (701) that contains enough information to reconstruct that inputimage. FIG. 8 illustrates six different pairs of, for example 48×48,input patches (701 of FIG. 7) and the corresponding reconstructedpatches (708 of FIG. 7), which were reconstructed using the trained CAE.

Once the CAE is trained, the decoder part (709 of FIG. 7) is removed andthe fully connected layer (710 of FIG. 7) becomes the output layer whichis used to generate encodings for unseen patches.

As a functionally significant stenosis is expected to have a localimpact on the myocardial blood perfusion, and consequently on thetexture characteristics of the contrast enhancement of hypoperfusedregions, the LV myocardium is divided into a number of spatiallyconnected clusters as described in step 606 of FIG. 6. Clustering isachieved, for instance, using the fast K-means algorithm (“Web-scalek-means clustering,” Proceedings of the 19^(th) international conferenceon World wide web, ACM, 2010, pp 1177-1178), based on the spatiallocation of the myocardial voxels. FIG. 9 shows the result is such aclustering method. Within a single cluster, a large deviation of anencoding likely indicates its inhomogeneity, and thereby the presence ofan abnormal myocardial tissue. Therefore, the standard deviation (STD)of each of the encodings over all voxels within a cluster is calculated.Thereafter, to describe the whole LV myocardium rather than itsclusters, the maxima of the standard deviations of each encoding overall clusters are used as features describing the LV myocardium of eachpatient. Furthermore, higher order statistical parameters can becalculated and used such as skewness, kurtosis or higher moments.Besides the use of the STD to reduce the high dimensionality of theencodings over all the voxels within a cluster, alternative ways ofcompressing the local encodings can be performed. For instance, by meansof restricted Boltz-mann machine (as for example presented by Bengio etal., “Representation Learning: A Review and New Perspectives,” IEEETrans. Pattern Anal. Mach. Intell. 35 (8), 2013, 1798-1828) or deepbelief networks (as for example taught by Lee at al., “Convolutionaldeep belief networks for scalable unsupervised learning of hierarchicalrepresentations,” Proceedings of the 26th Annual InternationalConference on Machine Learning, 2009, pp. 609-616). These generativeapproaches, which belong to the undirected graphical models, could beemployed to represent a group of voxel encodings by more compressed butyet descriptive representations.

The clustering can be performed using any clustering method. Anotherexample of clustering is by means of the The American Heart Association17-segment heart model (Cerqueira et al., “Standardized myocardialsegmentation and nomenclature for tomographic imaging of the heart. Astatement for healthcare professionals from the Cardiac ImagingCommittee of the Council on Clinical Cardiology of the American HeartAssociation,” Circulation Jan. 29, 2002; 105:539-542). The clusteringcan also be performed based on a patient specific 13-territory model, astaught by Cerci et al., “Aligning Coronary Anatomy and MycoardialPerfusion Territories: An Algorithm for the CORE320 Multicenter Study,”Circ Cardiovasc Imaging. 2012, 5:587-595. Finally, based on theextracted features, patients are classified into those with afunctionally significant coronary artery stenosis or those without.

With respect to clustering, coronary tree anatomy can be optionally usedto improve clustering of features. Different parts of the myocardium aresubtended by different sections of the coronary tree as can be seen inFIG. 10. When performing clustering of features it can occur thatcertain clusters contain voxels which receive their blood supply fromthe left side of the coronary tree (i.e. left anterior descending arteryor circumflex artery) as well as voxels which receive their blood supplyfrom the right side of the coronary tree (i.e. posterior descendingartery). When a stenosis is, for instance, only present in the posteriordescending artery of the coronary tree, this will affect the features ofthe voxels in the LV myocardium that are subtended by that section ofthe coronary tree. To avoid distortion of information within clusters,information regarding the coronary tree can be added to guide theclustering. The size of the cluster can for instance be limited to avoidterritories supplied by more than one main coronary (e.g. left coronaryartery, circumflex artery and right coronary artery). Such a method toidentify patient-specific blood supply territories may involvesegmentation of the myocardium, for instance by methods discusspreviously, and segmentation of the coronary tree centerline as forinstance by Metz et al., “Coronary centerline extraction from CTcoronary angiographic images using a minimum cost path approach,” MedPhys. 2009 December; 36(12):5568-79. Based on the patient specificsegmented myocardium and the patient specific segmented coronary tree,patient-specific blood supply territories can be calculated for instanceby the method as described by Zakkaroff et al., “Patient-specificcoronary blood supply territories for quantitative perfusion analysis,”Computer Methods in Biomechanics and Biomedical Engineering: Imaging &Visualization 2016 or by the method as taught by Termeer et al,“Patient-Specific Mappings between Myocardial and Coronary Anatomy,”Scientific Visualization: Advanced Concepts, 2010, page 196-209. Varioussegmentation and clustering methods may be utilized.

Within step 608 of FIG. 6, a feature vector is defined. For example, thefeature vector may comprise a series of factors, where each of thefactors has a value related to a characteristic of interest. Forexample, the factors may represent an amount of variation in thecharacteristic of interest over multiple clusters. The factors maycorrespond to separate encodings. For example, when 50 or more encodingsare utilized, the feature vector may comprise a series of 50 or morefactors. As one example, the characteristic of interest may representgrey scale of voxels or another characteristic acquired by a diagnosticimaging modality that concerns anatomical and/or functional aspects ofthe myocardium. The factor may represent a deviation (e.g., maximum orminimum standard deviation) in the characteristic of interest (e.g.,grey scale) over all or a subset of the clusters defined duringsegmentation. An example of a feature vector is given hereafter, wherethe feature vector comprises a series of variation factors for Nencodings and where the variation factors correspond to the maximumstandard deviation of the encodings resulting from the CAE within eachcluster as described before:

${{Feature}\mspace{14mu}{vector}} = \begin{pmatrix}{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} 1} \\{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} 2} \\\ldots \\{{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} N} - 1} \\{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} N}\end{pmatrix}$In above example of a feature vector, ‘the Maximum STD over all clustersfor encoding i’, where i=1, . . . N, is obtained by first computing thestandard deviations of a single encoding i over all voxels in a singlecluster; and subsequently determining the maximum of all standarddeviations of that encoding i over all clusters as described previously.

As another example, the factors may represent an intensity of thecharacteristic of interest over multiple segments of the myocardium. Forexample, when the 17 segment model as defined by the AHA is utilized,the feature vector may comprise a series of 17 factors. Morespecifically, the factors may represent a mean intensity of thecharacteristic of interest (e.g. grey scale) over all or a subset of thesegments defined during segmentation. An example of a feature vectorbased on a feature engineered Gaussian operator in combination with theAHA 17-segment model is given by:

${{Feature}\mspace{14mu}{vector}} = \begin{pmatrix}{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 1} \\{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 2} \\\ldots \\{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 16} \\{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 17}\end{pmatrix}$

In the above example, G_(intensity) is computed by a 2D or 3D Gaussianoperator with a specific kernel size, for instance 3 voxels. Both aboveexamples of feature vectors are based on local features in whichdimensionality reduction is performed. Additionally or alternatively,the feature vector may incorporate global features, for instancemyocardium volume and myocardium thickness. For example, the featurevector may include a series of factors having different types. Forexample, a subset of the factors in the series may represent meanintensity within segments, while another subset of the factors in theseries represent values indicative of myocardium volume, minimummyocardium thickness and maximum myocardium thickness. The followingexample shows a feature vector having factors of different types:

${{Feature}\mspace{14mu}{vector}} = \begin{pmatrix}{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 1} \\{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 2} \\\ldots \\{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 16} \\{{Mean}\mspace{14mu} G_{intensity}\mspace{14mu}{within}\mspace{14mu}{AHA}\mspace{14mu}{segment}\mspace{14mu} 17} \\{{Myocardium}\mspace{14mu}{volume}} \\{{Minimum}\mspace{14mu}{myocardium}\mspace{14mu}{thickness}} \\{{Maximum}\mspace{14mu}{myocardium}\mspace{14mu}{thickness}}\end{pmatrix}$

In case the CCTA dataset comprises multiple phases within the cardiaccycle, the myocardial features extracted from each phase, by performingthe steps described by 604, 605 and 606 of FIG. 6 for each phaseseparately, can be added to the feature vector. The same approach can beapplied in case the CCTA dataset consists of a multi-energy CCTAdataset. It might be beneficial to resample the multiphase CCTA datasetin the temporal domain to a fixed amount of cardiac cycles. With thisapproach, variations in the temporal resolution between differentmultiphase CCTA datasets can be resolved. Alternatively, instead ofadding each phase (or energy level) separately (by performing the stepsdescribed by 604, 605 and 606 of FIG. 6), a different deep learningnetwork architecture can be deployed in which the multiple phases (orenergy levels) are directly incorporated.

Besides the extracted features as described above (encodings andsegments), also additional information represented by step 607 of FIG.6, can be used to build up the feature vector. For example, demographicinformation such as weight, height, gender, etc. of the patient can beused as additional information for the patient classification byaddition as features to the feature vector.

The presence and amount of coronary artery calcification (see FIG. 11a )is a strong and independent predictor of cardiovascular events, whichcan be identified and quantified in CCTA as taught by Wolterink et al.“Automatic Coronary Artery Calcium Scoring in Cardiac CT AngiographyUsing Paired Convolutional Neural Networks,” Medical Image Analysis,2016. FIG. 11b , shows an example of detected coronary calcium within aCCTA dataset by using method as described by Wolterink et al. Thefeature vector can be expanded with information about the presence andextent of coronary calcium, per coronary artery (left anteriordescending artery, right coronary artery, and circumflex) or the totalcoronary tree. Additional information can be incorporated characterizingcoronary artery or other calcifications (e.g. in the aorta, heartvalves, pericardium)

Coronary events are also associated with the total plaque burden, whichincludes coronary plaque which are not necessary calcified. Beyond thedetection of calcified plaque, CCTA has promise in characterizing thetype of plaque (non-calcified and mixed) that is present. The totalamount of plaque can be determined by (semi) automated method thatdetects the inner and outer coronary vessel walls as for instance taughtby Dey et al., “Automated 3-dimensional quantification of noncalcifiedand calcified coronary plaque from coronary CT angiography,” CardiovascComput Tomogr. 2009, 3(6):372-382. The area between inner and outervessel walls is counted as plaque. The feature vector can be expandedwith the plaque burden as calculated by normalizing the volume of plaquewithin the vessel by the length of the vessel. Another example ofadditional information is protocol information. As described before,contrast material is administered to the patient prior to depiction ofthe heart and coronary arteries. Imaging is started once contrast mediumdensity surpasses a predefined threshold by either visual inspection, orby determination of contrast medium density in a predefined anatomicalstructure. For instance, the threshold used to start the CCTAacquisition, the anatomical structure used to assess the threshold, typeof CT scanner, moment within the cardiac cycle the CCTA acquisition isperformed (trigger time), and or the contrast medium administered to thepatient during acquisition can be used as additional information withinthe feature vector.

Additionally, if present, information obtained from any of the variousforms of dual-energy, spectral, multi-energy or photon-counting CT scancan be used as an additional feature for the classification. In CT,materials having different compositions can be represented by identicalpixel values on a CT image depending on the mass density of thematerial. Thus, the differentiation and classification of differenttissue types and contrast agents can be challenging. In a Dual-energyscan two CT datasets are acquired with different x-ray spectra, allowingthe differentiation of multiple materials. Not only anatomicalinformation is present but also information related to tissuecomposition. Therefore a better insight is available regarding the lumenand any ischemic tissue. Information regarding tissue compositionextracted from any of the various forms of dual-energy, spectral,multi-energy or photon-counting CT scan (for instance the presence ofischemic tissue, or contrast material) can be used as an additionalfeature.

Blood flow distribution within the myocardium and the location andextent of areas at risk in case of coronary artery disease are dependenton the distribution and morphology of intramural microvascular (vascularcrowns). The epicardial coronary arteries (right coronary artery, leftcoronary artery) distribute blood flow to different regions of the heartmuscle through the myocardium microvasculature. The myocardium can bedivided from epicardium to endocardium into three layers; subepicardial,mid-myocardium and subendocarial. The subendocardial layer is morevulnerable for ischemia and infarctions than the other layers as taughtby van den Wijngaard J P et al., “Model prediction of subendocardialperfusion of the coronary circulation in the presence of an epicardialcoronary artery stenosis,” Med Biol Eng Comput 2008, 46: 421-432.Optionally, this knowledge can be integrated in the feature vectorcalculation. For instance, the clusters (606) can be weighted with thespatial layer location, or a myocardium layer prediction model can beadded to the feature vector. Such a myocardium layer prediction modelcan be pre-generated based on for instance physiological experiments astaught by van Horssen et al., “Perfusion territories subtended bypenetrating coronary arteries increase in size and decrease in numbertoward the subendocardium,” Am J Physiol Heart Circ Physiol 2014, 306:H496-H504.

When an Electro Cardio Gram (ECG) is available from the patient, itscharacteristics can be used as additional information. An ECG is arepresentation of the electrical activity in the heart muscle. The ECGregisters the electric stimulus that causes the heart muscle cells tocontract. This stimulus travels from one muscle cell to the other. Thecardiac conduction system ensures that this is done in the rightsequence (i.e. atrium then ventricles). Typically, an ECG comprises offour segments, a P wave, a QRS complex, a T wave and a U wave. The Pwave represents atrial depolarization, the QRS complex representsventricular depolarization, the T wave represents ventricularrepolarization and the U wave represents papillary musclerepolarization. Changes in the structure of the heart and itssurroundings (including blood composition) change the patterns of thesefour segments. For instance, a heart attack or myocardial infarction isvisible in the ECG by an elevation of the ST segment as can be seen inFIG. 12. This is due to the fact that a section of the myocardium isischemic due to the heart attack. But also for instance chest pain(angina pectoris) and a heart block have ischemic markers that arecognizable on an ECG. When translating the 12-lead ECG into avectorcardiogram for instance as taught by Engels et al., “Thesynthesized vectorcardiogram resembles the measured vectorcardiogram inpatients with dyssynchronous heart failure,” J Electrocardiol;48(4):586-592, the electrical activity, as a vector, can be assessed inthe three principle directions. This allows more accurate and morerobust extracted of electrical parameters (Edenbrandt et al.,“Vectorcardiogram synthesized from a 12-lead ECG: Superiority of theinverse Dower matrix,” Journal of Electrocardiology, December 1988,21(4):361-7). For example, myocardial infarction and right ventricularhypertrophy assessed with synthesized vectorcardiogram are superior tothe corresponding 12-lead ECG criteria. Parameters extracted fromsynthesized vectorcardiogram are for instance QRS loop perimeter, QRSvector difference, area under the QRS complex, ST segment and T-wave inthe (X, Y, Z) leads; ST-T vector magnitude difference, T-wave vectormagnitude difference, and the spatial angle between the QRS complex andthe T-wave.

Other important feature which can be used as additional information arecardiac biomarkers. When blood is taken from the patient, levels ofcardiac biomarkers in the blood can be examined. These markers includeenzymes, hormones and proteins. Cardiac biomarkers show up in thepatient's blood after their heart has been under severe stress due toischemia, for instance due to a heart attack. The levels of thebiomarkers can be used to determine the size of the heart attack and howserious the effect of the heart attack is. Cardiac biomarkers are forinstance cardiac troponin and creatine kinase.

Furthermore, information regarding the presence of fat (adipose tissue)surrounding the heart or inside the heart, can be used as additionalinformation. This information can for instance be obtained using MR orCT data. Fat present directly around the heart (pericardial fat) maypredict narrowed arteries. People with fat in the area around the heartand under the breastbone in the chest, where it sits in close proximityto the heart, may face a higher risk of heart disease compared to peoplewho store fat in other areas. This is due to the fact that fatty tissuereleases inflammatory chemicals that may speed the development ofatherosclerosis.

Another feature that can be used is the shape of the myocardium. Forinstance, the presence of dilated cardiomyopathy (DCM). DCM is acondition in which the heart becomes enlarged and cannot pump bloodefficiently. DCM can be due to replacement of normal myocardium bydeposition of fibrous tissue in the myocardium, for instance subsequentto a previous myocardial infarction, or other diseases. An example ofDCM can be seen in FIG. 13. Information on the shape of the myocardiumand/or shape of the cardiac chambers can be obtained by principalcomponent analysis (PCA). PCA is a dimension-reduction method that canbe used to reduce a large set of variables to a small set that stillcontains most of the information in the large set, it is a mathematicalprocedure that transforms a number of (possibly) correlated variablesinto a (smaller) number of uncorrelated variables called principalcomponents. The first principal component accounts for as much of thevariability in the data as possible, and each succeeding componentaccounts for as much of the remaining variability as possible and eachsucceeding component in turn has the highest variance possible under theconstraint that it is orthogonal to the preceding components. An exampleof computing ventricular shape parameters by PCA is provided byMedrano-Gracia et al., “Left ventricular shape variation in asymptomaticpopulations: the Multi-Ethnic Study of Atherosclerosis,” J CardiovascMagn Reson. 2014 Jul. 30; 16:56. In this work Medrano-Gracia andcoworkers sought to establish the most important components of leftventricle shape and function present in an examined cohort. For currentapplication, the reference standard (601) database can be used to definethe principle components for the LV, RV and/or atriums by means of PCA.For this the desired cardiac structures needs to be segmented (only onetime) for the dataset used to define the principle components for thatcardiac structure. For this the method as described by step 302 of FIG.3 can be used, or any other method. Next the variation of the principlecomponents extracted from the examined CCTA dataset can be computed andadded as a feature to the feature vector. Alternatively, thesegmentation within step 302 of FIG. 3 can be performed by active shapemodel segmentation algorithms in which PCA is incorporated, or any othersegmentation algorithm in which PCA is incorporated. By using such analgorithm the principle components of the examined CCTA dataset areautomatically defined within this segmentation approach as for instancetaught by van Assen et al., “3D Active Shape Model Matching for LeftVentricle Segmentation in cardiac CT,” Phytochemistry January 2003,5032, or by D. Fritz et al., “Segmentation of the left and right cardiacventricle using a combined bi-temporal statistical model,” Proceedingsof SPIE—The International Society for Optical Engineering, March 2006,6141, D0110.1117/12.652991. Another feature based on PCA can be themyocardial tissue stiffness as taught by Wang et al., “Principalcomponent analysis used to derive patient-specific load-free geometryand estimate myocardial stiffness in the heart,”5th InternationalConference on Computational and Mathematical BiomedicalEngineering—CMBE2017. Another feature that can be used is myocardialstrain. Myocardial strain is a method for measuring regional or globaldeformation of the myocardium and can be used in the assessment ofconditions that impair myocardial function including ischemic heartdisease, hypertensive heart disease, dilated cardiomyopathy,hypertrophic cardiomyopathy, myocarditis, and infiltrativecardiomyopathies, cardiac dyssynchrony. Strain abnormalities develop inmost settings before overt clinical disease or even mild subclinicalabnormalities of ventricular ejection fraction and have prognosticsignificance, as do severity and progression of strain abnormalities inadvanced or treated disease. Myocardium strain can be added to thefeature vector for instance by the method as though by Wong et al.,“Myocardial strain estimation from CT: towards computer-aided diagnosison infarction identification,” SPIE Medical Imaging conference, March2015, DOI 10.1117/12.2081464.

Other features that can be used are end diastolic LV blood volume, endsystolic LV blood volume, ejection fraction, cardiac output, diameter ofascending aortic, present of bicuspid aortic valve, cardiac valveinsufficiency (mitral, aorta, tricuspid and/or pulmonary) and/or thecoronary tree dominance; left dominant, right dominant, balanced orsmall right/left dominant.

In additional to all the features regarding the LV myocardium also thesame information regarding the RV myocardium can be taken into accountfor the classification.

An example of a feature vector (step 608 of FIG. 6), considering themaximum standard deviation within each cluster as described before, andtaking some of the additional features into considerations, is given by:

${{Feature}\mspace{14mu}{vector}} = \begin{pmatrix}{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} 1} \\{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} 2} \\\ldots \\{{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} N} - 1} \\{{Maximum}\mspace{14mu}{STD}\mspace{14mu}{over}\mspace{14mu}{all}\mspace{14mu}{clusters}\mspace{14mu}{for}\mspace{14mu}{encoding}\mspace{14mu} N} \\{{Total}\mspace{14mu}{coronary}\mspace{14mu}{calcium}} \\{{Number}\mspace{14mu}{of}\mspace{14mu}{calcified}\mspace{14mu}{lesions}} \\{{Patient}\mspace{14mu}{blood}\mspace{14mu}{pressure}} \\{{Patient}\mspace{14mu}{age}} \\{{Amount}\mspace{14mu}{of}\mspace{14mu}{creatine}\mspace{14mu}{kinase}} \\{{Coronary}\mspace{14mu}{plaque}\mspace{14mu}{burden}}\end{pmatrix}$

Extracted features and features obtained from additional informationthat are present that can be used to classify a patient as havingfunctionally significant stenosis or not.

At step 609 of FIG. 6 the feature-perfusion classification model islearned by means of a supervised classifier. Several supervisedclassifiers could be used, for instance, a support vector machine (SVM)classifier. A SVM is a supervised machine learning classifier that canbe employed for both classification and regression purposes. SVMs arebased on the idea of finding a hyperplane (141, FIG. 14) that bestdivides a dataset into predefined classes (142, FIG. 14). As a simpleexample, for a classification task with only two features is illustratedin FIG. 14. During training of the SVM, a hyperplane that best separatessamples of two classes is found by maximizing the margin around thedecision boundary while minimizing the number of training samples withinthe margin (FIG. 14). The margin (143, FIG. 14) is determined by thesupport vectors (144, FIG. 14) i.e. training samples that lie on themargin. Intuitively, a good separation is achieved by the hyperplanethat has the largest distance to the nearest training-data point of anyclass. In other words, the distance between the hyperplane and thenearest support vector from either set is known as the margin. The goalof SVM is to find a hyperplane with the greatest possible margin betweenthe hyperplane and any point (support vector) within the training set.

Other kinds of classifiers may include neural networks, Bayesianclassifiers, Tree Ensembles (e.g., random Forests) (Kotsiantis et al,“Supervised Machine Learning: A Review of Classification Techniques,”Informatica 31, 2007, 249-268).

To be able to use a supervised (SVM) classifier, reference data must bepresent that can be used as a reference standard. The reference standardis a database from multiple patients (step 601). Each set within thisdatabase contains a) contrast enhanced CT datasets (step 602) withbelonging b) reference value (step 603).

In a preferred embodiment, the reference value (603), representing afluidodynamic parameter, is an invasive fractional flow reserve (FFR)measurement as performed during X-ray angiography which belongs to thecontrast enhanced CT dataset (602). For example, FFR is measured with acoronary pressure guidewire at maximal hyperemia induced by intravenousadenosine. During X-ray angiography the FFR wire is placed as distallyas possible in the target vessel and FFR is assessed by means of amanual or automatic pullback in the distal part of the target vessel.Finally, the FFR wire is retrieved at the level of the guiding catheterto achieve a FFR value of 1.00 in order to assess the quality of themeasurement performed. When multiple FFR measurements are available dueto repeated measurements or multiple stenosis, the minimal value istaken as the standard of reference. The reference value (603) can be anyparameter which links the patient specific CCTA datasets to myocardialischemia of that patient. For instance, the reference value (603) can bethe measured coronary flow reserve or the index of microcirculatoryresistance which provides a measurement of the minimum achievablemicrocirculatory resistance in a target coronary artery territory,enabling a quantitative assessment of the microvascular integrity. Otherexamples of different parameters for the reference value (603) are theoccurrence of major adverse cardiac events (MACE) within a predefinedamount of time after acquisition of the CCTA dataset, or if the patientunderwent revascularization within a predefined amount of time afteracquisition of the CCTA dataset, or the results of cardiac stress test,the results of myocardial magnetic resonance imaging (MM) perfusion,SPECT, PET, CT perfusion, or ultrasound.

Using a database of reference values (603), which corresponds to theused CCTA dataset (602), each reference value (603) is marked asbelonging to one of two classes, for instance “functionally significantstenosis present” (invasive FFR<for instance 0.8) or “no significantstenosis present” (invasive FFR>for instance 0.8) (the known labels),the SVM classifier learns to separate the different classes. First, eachtraining sample (e.g. CCTA dataset) is represented as a point in ann-dimensional feature space, where n is the number of computed features(e.g. the number of features in the feature vector, the result of step608 of FIG. 6). For all reaming CCTA cases in the database (602), such afeature vector is computed. All training samples (e.g. CCTA cases in thedatabase) are used to train the chosen classifier. In the trainingphase, the SVM classifier finds the hyperplane that makes the bestseparation between the classes i.e. by finding a hyperplane separatingtwo classes with the largest margin as illustrated in FIG. 14.

SVM is in nature a two-class classifier. Nevertheless, multi-classclassification, i.e. classification in multiple classes, can beperformed by e.g. performing multiple 2-class classifications (e.g.chosen class vs. all remaining classes or between every pair ofclasses—one vs one). Hence, the feature-perfusion classification modelcan be trained to recognize multiple classes, for example “nofunctionally significant stenosis present”, “mild functionallysignificant stenosis present” or “severe functionally significantstenosis present”, or any categories chosen based on the reference value(step 603 of FIG. 6). When the reference value (FIG. 6, 603) is aninvasive FFR measurement, above classification can be achieved using forinstance the following invasive FFR threshold values:

-   -   Invasive FFR>0.9, “no functionally significant stenosis present”    -   Invasive FFR between 0.7 and 0.8 “mild functionally significant        stenosis present”    -   Invasive FFR<0.7, “severe functionally significant stenosis        present”

Once the system is trained, new unseen CCTA datasets are classified intothe classes as defined during the training phase, which is furtherexplained by the flowchart of FIG. 15.

FIG. 15 illustrates a framework for implementing the prediction phase,to classify the severity of vessel obstruction(s) within unseen CCTAdatasets; to determine the presents of functional significant stenosisin one or more coronary arteries from a CCTA dataset. The unseen CCTAdataset is represented by block 151 of FIG. 15.

In block 152 the myocardium segmentation takes place, according to oneof the methods as described previously by block 302 of FIG. 3.

Within block 303 of FIG. 15 the computing of the features takes place,and is identical as described previously for block 303 of FIG. 6. Block153 of FIG. 15 describes the feature extraction and is identical to themethod as described by step 605 of FIG. 6. In a preferred embodiment,the feature extraction during the learning phase, as performed by block303 of FIG. 6, is performed by means of CAE. Within the prediction phasethe architecture of the CAE is slightly different. As describe before inblock 605 and referring to FIG. 6, after the CAE is trained, the decoderpart can be removed and the fully connected layer becomes the outputlayer which is used to generate encodings for unseen patches. An exampleof such CAE architecture as used in the prediction phase is presented inFIG. 16. This figure shows the CAE architecture for the prediction phasefor the example as presented by FIG. 7. Step 154 within FIG. 15describes the clustering and is identical to the method as described instep 606 of FIG. 6. Block 155 describes the additional information andis identical to the methods as described in step 607 of FIG. 6. Next thefeature vector is generated within step 156 by the same method asperformed during the learning as describe previously by block 608 ofFIG. 6. In case additional information was used in the learning phase,the same methods are performed within step 175 during the predictionphase.

Step 157 of FIG. 15 represents the feature-perfusion classificationmodel as learned during the learning phase as described by step 609 ofFIG. 6.

Finally, in step 158, the classifier assigns new unseen CCTA datasetsinto the categories as defined during the training phase. Thisclassifier is the same classifier as used in block 609 in FIG. 6. Withinthe prediction phase, unseen CCTA dataset are mapped by step 158 of FIG.15 in the n-dimensional feature space and its location in this featurespace with respect to the hyperplane determines its class label. Thisclassification results in an assessment of the severity that one or morecoronary obstructions impedes oxygen delivery to the heart muscle and isrepresented by step 159 of FIG. 15. FIG. 17 shows a visualrepresentation of the classifier. The result of step 156 of FIG. 15,representing the feature vector of the unseen image, is the input (171)of the classifier. Label 172 of FIG. 17 represents the feature-perfusionclassification model (FIG. 15, 156), as learned during the learningphase as described by step 609 of FIG. 6. Label 173 of FIG. 17 representthe output of the classifier (FIG. 16, 159) incase two classes arelearned; positive meaning one or more functionally significant coronarylesions present, and negative meaning no functionally significantcoronary lesion present.

In case when dealing with multiphase CCTA datasets or multi energy CCTAdatasets by the approach as described within step 608 of FIG. 6, thefeature vector can be expanded with features from each phase or energylevel. In this case, multiple feature-perfusion models can be learnedfor each of these sets (which, for example, can be single phase CCTAdatasets, multiphase CCTA datasets, multi energy CCTA datasets). Thismeans that during the training phase (FIG. 6) as well as during theprediction phase (FIG. 15), the feature-perfusion model of thatparticular CCTA dataset needs to be selected. This can be done forinstance by examination of the information obtained through the“headers” of the DICOM file format. Also, multiple feature-perfusionmodels can be generated depended on the amount of additional informationas described in step 607 of FIG. 6 (e.g. shape parameters, ECGparameters, etc.), meaning that in cases where not all additionalfeatures are available, the correct feature-perfusion model can beselected from the multiple models.

The present disclosure mainly describes the organ of interest as themyocardium and the vessels being the coronary arteries. The skilledperson would appreciate that this teaching can be equally extended toother organs. For instance, the organ of interest can be the kidneywhich is perfused by the renal arteries, or (parts) of the brain asperfused by the intracranial arteries. Furthermore, the presentdisclosure refers to CCTA datasets (in several forms). The skilledperson would appreciate that this teaching can be equally extended toother imaging modalities, for instance rotational angiography, MRI,SPECT, PET, Ultrasound, X-ray, or the like.

The embodiment of this disclosure can be used on a standalone system orincluded directly in, for instance, a computed tomography (CT) system.FIG. 18 illustrates an example of a high-level block diagram of acomputed tomography (CT) system. In this block diagram the embodiment isincluded as an example how the present embodiment could integrate insuch a system.

Portions of the system (as defined by various functional blocks) may beimplemented with dedicated hardware, analog and/or digital circuitry,and/or one or more processors operating program instructions stored inmemory.

The most common form of computed tomography is X-ray CT, but many othertypes of CT exist, such as dual-energy, spectral, multi-energy orphoton-counting CT. Also, positron emission tomography (PET) andsingle-photon emission computed tomography (SPECT) or combined with anyprevious form of CT.

The CT system of FIG. 18 describes an X-ray CT system. In an X-ray CTsystem an X-ray system moves around a patient in a gantry and obtainsimages. Through use of digital processing a three-dimensional image isconstructed from a large series of two-dimensional angiographic imagestaken around a single axis of rotation.

For a typical X-ray CT system 120 an operator positions a patient 1200on the patient table 1201 and provides input for the scan using anoperating console 1202. The operating console 1202 typically comprisesof a computer, a keyboard/foot paddle/touchscreen and one or multiplemonitors.

An operational control computer 1203 uses the operator console input toinstruct the gantry 1204 to rotate but also sends instructions to thepatient table 1201 and the X-ray system 1205 to perform a scan.

Using a selected scanning protocol selected in the operator console1202, the operational control computer 1203 sends a series of commandsto the gantry 1204, the patient table 1201 and the X-ray system 1205.The gantry 1204 then reaches and maintains a constant rotational speedduring the entire scan. The patient table 1201 reaches the desiredstarting location and maintains a constant speed during the entire scanprocess.

The X-ray system 1205 includes an X-ray tube 1206 with a high voltagegenerator 1207 that generates an X-ray beam 1208.

The high voltage generator 1207 controls and delivers power to the X-raytube 1206. The high voltage generator 1207 applies a high voltage acrossthe vacuum gap between the cathode and the rotating anode of the X-raytube 1206.

Due to the voltage applied to the X-ray tube 1206, electron transferoccurs from the cathode to the anode of the X-ray tube 1206 resulting inX-ray photon generating effect also called Bremsstrahlung. The generatedphotons form an X-ray beam 1208 directed to the image detector 1209.

An X-ray beam 1208 comprises of photons with a spectrum of energies thatrange up to a maximum determined by among others the voltage and currentsubmitted to the X-ray tube 1206.

The X-ray beam 1208 then passes through the patient 1200 that lies on amoving table 1201. The X-ray photons of the X-ray beam 1208 penetratethe tissue of the patient to a varying degree. Different structures inthe patient 1200 absorb different fractions of the radiation, modulatingthe beam intensity.

The modulated X-ray beam 1208′ that exits from the patient 1200 isdetected by the image detector 1209 that is located opposite of theX-ray tube.

This image detector 1209 can either be an indirect or a direct detectionsystem.

In case of an indirect detection system, the image detector 1209comprises of a vacuum tube (the X-ray image intensifier) that convertsthe X-ray exit beam 1208′ into an amplified visible light image. Thisamplified visible light image is then transmitted to a visible lightimage receptor such as a digital video camera for image display andrecording. This results in a digital image signal.

In case of a direct detection system, the image detector 1209 comprisesof a flat panel detector. The flat panel detector directly converts theX-ray exit beam 1208′ into a digital image signal.

The digital image signal resulting from the image detector 1209 ispassed to the image generator 1210 for processing. Typically, the imagegeneration system contains high-speed computers and digital signalprocessing chips. The acquired data are preprocessed and enhanced beforethey are sent to the display device 1202 for operator viewing and to thedata storage device 1211 for archiving.

In the gantry the X-ray system is positioned in such a manner that thepatient 1200 and the moving table 1201 lie between the X-ray tube 1206and the image detector 1209.

In contrast enhanced CT scans, the injection of contrast agent must besynchronized with the scan. The contrast injector 1212 is controlled bythe operational control computer 1203.

For FFR measurements an FFR guidewire 1213 is present, also adenosine isinjected by an injector 1214 into the patient to induce a state ofmaximal hyperemia.

An embodiment of the present application is implemented by the X-ray CTsystem 120 of FIG. 18 as follows. A clinician or other user acquires aCT scan of a patient 1200 by selecting a scanning protocol using theoperator console 1202. The patient 1200 lies on the adjustable table1201 that moves at a continuous speed during the entire scan controlledby the operational control computer 1203. The gantry 1204 maintains aconstant rotational speed during the entire scan

Multiple two-dimensional X-ray images are then generated using the highvoltage generator 1207, the X-ray tube 1206, the image detector 1209 andthe digital image generator 1210 as described above. This image is thenstored on the hard drive 1211. Using these X-ray images, athree-dimensional image is constructed by the image generator 1210.

The general processing unit 1215 uses the three-dimensional image toperform the classification as described above.

There have been described and illustrated herein several embodiments ofa method and apparatus for automatically identify patients withfunctionally significant stenosis, based on the information extractedfrom a single CCTA image only.

While particular embodiments of the present application have beendescribed, it is not intended that the present application be limitedthereto, as it is intended that the present application be as broad inscope as the art will allow and that the specification be read likewise.

For example, multi-phase CCTA datasets can be used, functionalassessment of renal arties in relation to the perfused kidney can beassess based on the methodology disclosed, the data processingoperations can be performed offline on images stored in digital storage,such as a PACS or VNA in DICOM (Digital Imaging and Communications inMedicine) format commonly used in the medical imaging arts. It willtherefore be appreciated by those skilled in the art that yet othermodifications could be made to the provided application withoutdeviating from its spirit and scope as claimed.

The embodiments described herein may include a variety of data storesand other memory and storage media as discussed above. These can residein a variety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network. In a particular set of embodiments,the information may reside in a storage-area network (“SAN”) familiar tothose skilled in the art.

Similarly, any necessary files for performing the functions attributedto the computers, servers or other network devices may be stored locallyand/or remotely, as appropriate.

Where a system includes computerized devices, each such device caninclude hardware elements that may be electrically coupled via a bus,the elements including, for example, at least one central processingunit (“CPU” or “processor”), at least one input device (e.g., a mouse,keyboard, controller, touch screen or keypad) and at least one outputdevice (e.g., a display device, printer or speaker). Such a system mayalso include one or more storage devices, such as disk drives, opticalstorage devices and solid-state storage devices such as random-accessmemory (“RAM”) or read-only memory (“ROM”), as well as removable mediadevices, memory cards, flash cards, etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.) and working memory asdescribed above.

The computer-readable storage media reader can be connected with, orconfigured to receive, a computer-readable storage medium, representingremote, local, fixed and/or removable storage devices as well as storagemedia for temporarily and/or more permanently containing, storing,transmitting and retrieving computer-readable information. The systemand various devices also typically will include a number of softwareapplications, modules, services or other elements located within atleast one working memory device, including an operating system andapplication programs, such as a client application or web browser.

It should be appreciated that alternate embodiments may have numerousvariations from that described above. For example, customized hardwaremight also be used and/or particular elements might be implemented inhardware, software (including portable software, such as applets) orboth.

Further, connection to other computing devices such as networkinput/output devices may be employed.

Various embodiments may further include receiving, sending, or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-readable medium. Storage media and computerreadable media for containing code, or portions of code, can include anyappropriate media known or used in the art, including storage media andcommunication media, such as, but not limited to, volatile andnon-volatile, removable and non-removable media implemented in anymethod or technology for storage and/or transmission of information suchas computer readable instructions, data structures, program modules orother data, including RAM, ROM, Electrically Erasable ProgrammableRead-Only Memory (“EEPROM”), flash memory or other memory technology,Compact Disc Read-Only Memory (“CD-ROM”), digital versatile disk (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices or any other medium whichcan be used to store the desired information and which can be accessedby the system device. Based on the disclosure and teachings providedherein, a person of ordinary skill in the art will appreciate other waysand/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the presentapplication as set forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit thepresent application to the specific form or forms disclosed, but on thecontrary, the intention is to cover all modifications, alternativeconstructions and equivalents falling within the spirit and scope of thepresent application, as defined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including”and “containing” are to be construed as open ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected,” when unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to orjoined together, even if there is something intervening.

Recitation of ranges of values herein are merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range, unless otherwise indicated herein and eachseparate value is incorporated into the specification as if it wereindividually recited herein. The use of the term “set” (e.g., “a set ofitems”) or “subset” unless otherwise noted or contradicted by context,is to be construed as a nonempty collection comprising one or moremembers.

Further, unless otherwise noted or contradicted by context, the term“subset” of a corresponding set does not necessarily denote a propersubset of the corresponding set, but the subset and the correspondingset may be equal.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. Processes described herein (or variationsand/or combinations thereof) may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs or one or more applications) executing collectively onone or more processors, by hardware or combinations thereof. The codemay be stored on a computer-readable storage medium, for example, in theform of a computer program comprising a plurality of instructionsexecutable by one or more processors. The computer-readable storagemedium may be non-transitory.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the presentapplication. Variations of those preferred embodiments may becomeapparent to those of ordinary skill in the art upon reading theforegoing description. The inventors expect skilled artisans to employsuch variations as appropriate and the inventors intend for embodimentsof the present disclosure to be practiced otherwise than as specificallydescribed herein.

Accordingly, the scope of the present disclosure includes allmodifications and equivalents of the subject matter recited in theclaims appended hereto as permitted by applicable law. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the scope of the present disclosure unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications and patents,cited herein are hereby incorporated by reference to the same extent asif each reference were individually and specifically indicated to beincorporated by reference and were set forth in its entirety herein.

The invention claimed is:
 1. A method for assessing a severity of vesselobstruction, comprising: a) obtaining a contrast enhanced volume imagedataset for a target organ that includes at least one of a myocardium ora coronary artery, wherein at least a portion of the volume image dataset is segmented into data segments; b) obtaining features indicative ofa characteristic experienced by the data segments of the target organ;c) obtaining a feature-perfusion classification (FPC) model derived froma training set of perfused organs, wherein the FPC model includes arelationship between training features and a reference fluid dynamicparameter for corresponding data segments of the training set ofperfused organs, wherein the reference fluid dynamic parameter comprisesi) an invasive fractional flow reserve measurement, ii) an index ofmicrocirculatory resistance, iii) an instantaneous wave-free ratiomeasurement, or iv) a coronary flow reserve measurement; d) classifyingof the data segments based on the features obtained and based on the FPCmodel; and e) providing an output related to vessel obstruction based onthe classification of the data segments.
 2. The method according toclaim 1, wherein the characteristic represents an amount of perfusionexperience by the data segments.
 3. The method according to claim 1,wherein the output is indicative of a severity of the vesselobstruction.
 4. The method according to claim 1, wherein the a)obtaining the contrast enhanced volume image data set further comprises:a1) segmenting at least a portion of the myocardium of the target organfrom the volume image data set; a2) determining an anatomic model of oneor more coronary arteries from the volume image data set; and a3)dividing the myocardium of the target organ into wall regions using theanatomic model, wherein at least a portion of the data segmentscorresponds to the wall regions.
 5. The method according to claim 1,wherein: the features are texture and/or morphologic features, the organis the heart and the vessels the coronary arteries.
 6. The methodaccording claim 1, wherein: the features are determined using at leastone of a convolutional auto-encoder, a Gaussian filter, a transmuralperfusion ratio, a Haralick feature, and thickness or shape of thetarget organ.
 7. The method according to claim 1, wherein: the obtainingof b) includes extracting, for each of the data segments, a featurevector that comprises multiple factors that are measured or extractedfrom the corresponding data segment, wherein the multiple factorsdescribe or characterize a nature of the corresponding data segments. 8.The method according to claim 1, wherein: the obtaining of b) includesextracting, for each of the data segments, a feature vector thatcomprises of secondary information comprising one or more of thefollowing parameters: coronary tree anatomy, demographic information ofthe patient, coronary artery calcification, coronary plaque, plaqueburden, spectral multi-energy or photon counting, ECG parameters,cardiac biomarkers, adipose tissue surrounding or within the heart, orshape of myocardium.
 9. The method according to claim 1, wherein: theclassifying of d) is performed using at least one of a support vectormachine, a neural network, a random forest, and a Bayesian or a TreeEnsembles classifier.
 10. The method according to claim 1, wherein: theFPC model is obtained from a database of contrast enhanced volume imagedata sets and associated training feature vectors extracted from thecontrast enhanced volume image data sets, the training feature vectorsincluding known labels; and the classifying of d) utilizes amachine-learning algorithm that is trained based on the known labels,the machine-learning algorithm classifying the data segments based onthe features.
 11. The method according to claim 1, further comprising:implementing a training phase to form the FPC model that classifiestraining features for the training set of perfused organs from contrastenhanced volume image datasets of the organ of the training set and areference fluid dynamic parameter related to a vessel or vesselsperfusing the organs, the training phase comprising: i) providingcontrast enhanced volume image datasets of each of the organs in thetraining set; ii) segmenting at least a portion of the volume image dataset of i) into data segments; iii) analyzing the data segments of ii) toextract training features that are indicative of an amount of perfusionexperiences by wall regions of the organs of the training set; and iv)classifying the training features of iii) relative to reference fluiddynamic parameters indicative of baseline amounts of vessel perfusionfor corresponding regions of the training set of perfused organs to formthe FPC model.
 12. The method according to claim 11, wherein: thesegmenting of ii) includes segmenting at least a portion of themyocardium of the target organs from the volume image data set anddetermining an anatomic model of one or more coronary arteries from thevolume image data to divide the myocardium of the target organ into datasegments.
 13. The method according to claim 11, wherein: the analyzingof iii) clusters extracted training features before performing theclassifying of iv).
 14. The method according to claim 1, wherein: theobtaining of b) clusters the obtained features before performing theclassifying of d).
 15. The method according to claim 1, wherein: theobtaining of b) extracts a feature vector comprising a series offactors, where each of the factors has a value representing an amount ofvariation in a characteristic of interest over multiple clusters. 16.The method according to claim 1, wherein: the features of b) areencodings determined in a unsupervised manner using a convolutionauto-encoder and the clusters are obtained by dividing the myocardiuminto spatially connected regions.
 17. The method according to claim 1,wherein the features are calculated using a statistical parameterextracted from the encodings within clusters.
 18. A method utilizing afeature-perfusion classification (FPC) model that classifies trainingfeatures for assessing a severity of vessel obstruction, the methodcomprising: a) obtaining a contrast enhanced volume image dataset for atraining perfused organ that includes at least one of a myocardium or acoronary artery, wherein at least a portion of the volume image data setfor the training perfused organ is segmented into data segments; b)obtaining training features indicative of a characteristic experiencedby the data segments of the training perfused organ; c) classifying thetraining features of b) relative to reference fluid dynamic parametersindicative of baseline amounts of vessel perfusion for correspondingdata segments of the training perfused organ to form the FPC model,wherein the FPC model includes a relationship between the trainingfeatures and the reference fluid dynamic parameter for correspondingwall regions of the training perfused organ, wherein the reference fluiddynamic parameter comprises at least one of i) an invasive fractionalflow reserve measurement, ii) an index of microcirculatory resistance,iii) an instantaneous wave-free ratio measurement, or iv) a coronaryflow reserve measurement; d) obtaining a contrast enhanced volume imagedataset for a target organ that includes at least one of a myocardium ora coronary artery, wherein at least a portion of the volume image dataset for the target organ is segmented into data segments; e) obtainingfeatures indicative of an amount of perfusion experienced by datasegments of the target organ; f) classifying the data segments of d)based on the features extracted in e) and the FPC model of c); and g)providing, as an output, a prediction related to vessel obstructionbased on the classification of the data segments in f).
 19. The methodaccording to claim 18, wherein the characteristic represents an amountof perfusion experience by the data segments.
 20. The method accordingto claim 18, wherein the output is indicative of a severity of thevessel obstruction.
 21. The method according to claim 18, wherein the a)obtaining the contrast enhanced volume image data for a trainingperfused set further comprises: a1) segmenting at least a portion of themyocardium of the target organ from the volume image data set; a2)determining the anatomic model of one or more coronary arteries from thevolume image data set; and a3) dividing the myocardium of the targetorgan into wall regions using the anatomic model, wherein at least aportion of the data segments corresponds to the wall regions; andwherein the d) obtaining the contrast enhanced volume image data setfurther comprising: d1) segmenting at least a portion of the myocardiumof the target organ from the volume image data set; d2) determining theanatomic model of one or more coronary arteries from the volume imagedata set; and d3) dividing the myocardium of the target organ into wallregions using the anatomic model, wherein at least a portion of the datasegments corresponds to the wall regions.
 22. The method according toclaim 18, wherein: the obtaining of b) clusters the training featuresbefore performing the classifying of c).
 23. A system for assessing aseverity of vessel obstruction, comprising: memory configured to store acontrast enhanced volume image dataset for a target organ that includesat least one of a myocardium or a coronary artery, wherein at least aportion of the volume image data set is segmented into data segments;and one or more processors that, when executing program instructionsstored in the memory, are configured to: a) obtain features indicativeof a characteristic experienced by the data segments of the targetorgan; b) obtain a feature-perfusion classification (FPC) model derivedfrom a training set of perfused organs, wherein the FPC model includes arelationship between training features and a reference fluid dynamicparameter for corresponding data segments of the training set ofperfused organs, wherein the reference fluid dynamic parameter comprisesi) an invasive fractional flow reserve measurement, ii) an index ofmicrocirculatory resistance, iii) an instantaneous wave-free ratiomeasurement, or iv) a coronary flow reserve measurement; c) classify thedata segments based on the features obtained and based on the FPC model;and d) provide an output relative to vessel obstruction based on theclassification of the data segments.
 24. The system according to claim23, wherein the output is indicative of a severity of the vesselobstruction.
 25. The system according to claim 23, wherein the one ormore processors are configured to segment the contrast enhanced volumeimage data into data segments set further comprising: a1) segmenting atleast a portion of the myocardium of the target organ from the volumeimage data set; a2) determining the anatomic model of one or morecoronary arteries from the volume image data set; and a3) dividing themyocardium of the target organ into wall regions using the anatomicmodel, wherein at least a portion of the data segments corresponds tothe wall regions.
 26. The system according to claim 23, wherein: thecharacteristic represents an amount of perfusion experience by the datasegments, the organ is the heart and the vessels the coronary arteries.27. The system according to claim 23, wherein: the features aredetermined using at least one of a convolutional auto-encoder in aunsupervised manner, a Gaussian filter, a transmural perfusion ratio, aHaralick feature, and a thickness or shape of the target organ.
 28. Thesystem according to claim 23, wherein: the one or more processors areconfigured to obtain the features in a) by extracting, for each of thedata segments, a feature vector that comprises multiple factors that aremeasured or extracted from the corresponding data segment, wherein themultiple factors describe or characterize a nature of the correspondingdata segments.
 29. The system according to claim 23, wherein: the one ormore processors are configured to obtain the features in a) byextracting, for each of the data segments, a feature vector thatcomprises of secondary information comprising one or more of thefollowing parameters: anatomic model of the coronary arteries,demographic information of the patient, coronary artery calcification,coronary plaque, plaque burden, spectral multi-energy or photoncounting, ECG parameters, cardiac biomarkers, adipose tissue surroundingor within the heart, or shape of myocardium.
 30. The system according toclaim 28, wherein: the one or more processors are configured to performthe classification of c) using at least one of a support vector machine,a neural network, a random forest, and a Bayesian or a Tree Ensemblesclassifier.